Dynamic reference motion vector coding mode

ABSTRACT

Techniques are described to use a reference motion vector to reduce the amount of bits needed to encode motion vectors for inter prediction. One method includes identifying a candidate motion vector used to inter predict each of a plurality of previously coded blocks to define a plurality of candidate motion vectors, determining a distance between the previously decoded block associated with the candidate motion vector and a current block, determining a popularity value for each candidate motion vector based on a set of previously coded pixels associated with the candidate motion vector, and ranking the candidate motion vectors according to a priority value for each candidate motion vector. The priority value, based on the distance and the popularity value associated with the candidate motion vector, is used to select a reference motion vector from the candidate motion vectors that is used to encode the motion vector for the current block.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/288,553, filed Jan. 29, 2016, which is incorporated herein in itsentirety by reference.

TECHNICAL FIELD

This disclosure relates to video encoding and decoding.

BACKGROUND

Digital video streams may represent video using a sequence of frames orstill images. Digital video can be used for various applicationsincluding, for example, video conferencing, high definition videoentertainment, video advertisements, or sharing of user-generatedvideos. A digital video stream can contain a large amount of data andconsume a significant amount of computing or communication resources ofa computing device for processing, transmission or storage of the videodata. Various approaches have been proposed to reduce the amount of datain video streams, including compression and other encoding techniques.

SUMMARY

This disclosure relates generally to encoding and decoding video dataand more particularly relates to video coding using reference motionvectors. The teachings herein can reduce the number of bits required toencode motion vectors for inter prediction. One method for encoding avideo stream having a plurality of blocks including a current blockdescribed herein includes identifying, for each previously coded blockof a plurality of previously coded blocks of the video stream, acandidate motion vector used to inter predict the previously coded blockto define a plurality of candidate motion vectors; determining, using aprocessor, for each candidate motion vector of the plurality ofcandidate motion vectors, a distance between the previously coded blockassociated with the candidate motion vector and a current block of acurrent frame; determining, using the processor, for each candidatemotion vector of the plurality of candidate motion vectors, a popularityvalue based on a set of previously coded pixels associated with thecandidate motion vector; ranking each candidate motion vector of theplurality of candidate motion vectors according to a priority value foreach candidate motion vector, the priority value based on the distanceand popularity value of the candidate motion vector; and selecting,based on the priority values, a reference motion vector from theplurality of candidate motion vectors; and encoding a motion vector forthe current block using the reference motion vector.

Another method described herein is a method for decoding an encodedvideo bitstream, including determining, from bits included in theencoded video bitstream, a reference frame on a condition that a motionvector for a current block to be decoded was encoded using a referencemotion vector and whether the motion vector for the current block to bedecoded was encoding using single prediction or compound prediction, thecurrent block comprising one of a plurality of blocks of a current frameof the encoded video bitstream; identifying, for each previously decodedblock of a plurality of previously decoded blocks, a candidate motionvector used to inter predict the previously decoded block to define aplurality of candidate motion vectors; determining, using a processor,for each candidate motion vector of the plurality of candidate motionvectors, a distance between the previously coded block associated withthe candidate motion vector and the current block; determining, usingthe processor, for each candidate motion vector of the plurality ofcandidate motion vectors, a popularity value based on a set ofpreviously coded pixels associated with the candidate motion vector;ranking each candidate motion vector of the plurality of candidatemotion vectors according to a priority value for each candidate motionvector, the priority value based on the distance and the popularityvalue associated with the candidate motion vector; determining apredicted motion vector from the plurality of candidate motion vectors,the predicted motion vector associated with the candidate motion vectorhaving the highest popularity value; determining a first effectivemotion vector and a second effective motion vector, the first effectivemotion vector and the second effective motion vector associated with thecandidate motion vectors having the shortest distance; selecting thereference motion vector from the predicted motion vector, the firsteffective motion vector, and the second effective motion vector; anddecoding a motion vector for the current block using the referencemotion vector.

An example of an apparatus for encoding a video stream described hereinincludes a memory and a processor. The processor is configured toexecute instructions stored in the memory to identify, for eachpreviously coded block of a plurality of previously coded blocks, acandidate motion vector used to inter predict the previously coded blockto define a plurality of candidate motion vectors; determine, for eachcandidate motion vector of the plurality of candidate motion vectors, adistance between the previously coded block associated with thecandidate motion vector and a current block of a current frame;determine, for each candidate motion vector of the plurality ofcandidate motion vectors, a popularity value based on a set ofpreviously coded pixels associated with the candidate motion vector;rank each candidate motion vector of the plurality of candidate motionvectors according to a priority value for each candidate motion vector,the priority value based on the distance and popularity value associatedwith the candidate motion vector; select, based on the priority values,a reference motion vector from the plurality of candidate motionvectors; and encode a motion vector for the current block using thereference motion vector.

Variations in these and other aspects of the disclosure will bedescribed in additional detail hereafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The description herein makes reference to the accompanying drawingswherein like reference numerals refer to like parts throughout theseveral views, and wherein:

FIG. 1 is a schematic of a video encoding and decoding system inaccordance with implementations of this disclosure;

FIG. 2 is a diagram of an example video stream to be encoded and decodedin accordance with implementations of this disclosure;

FIG. 3 is a block diagram of a video compression system in accordancewith implementations of this disclosure;

FIG. 4 is a block diagram of a video decompression system in accordancewith implementations of this disclosure;

FIG. 5 is a flow diagram of a process for encoding a video stream usingreference motion vectors in accordance with an implementation of thisdisclosure;

FIG. 6 is a diagram of a frame including a current block used to explainthe process of FIG. 5;

FIG. 7 is a diagram of the current block of FIG. 6 and a set ofpreviously coded pixel;

FIG. 8 is a diagram of a set of predicted pixels for the set ofpreviously coded pixels of FIG. 7;

FIG. 9 is a flow diagram of a process for decoding an encoded videostream using reference motion vectors in accordance with implementationsof this disclosure;

FIG. 10 is a diagram of a series of frames of a first video stream inaccordance with an implementation of this disclosure; and

FIG. 11 is a diagram of a series of frames of a second video stream inaccordance with an implementation of this disclosure.

DETAILED DESCRIPTION

Compression schemes related to coding video streams may include breakingeach image into blocks and generating a digital video output bitstreamusing one or more techniques to limit the information included in theoutput. A received bitstream can be decoded to re-create the blocks andthe source images from the limited information. Encoding a video stream,or a portion thereof, such as a frame or a block, can include usingtemporal and spatial similarities in the video stream to improve codingefficiency. For example, a current block of a video stream may beencoded based on a previously encoded block in the video stream bypredicting motion and color information for the current block based onthe previously encoded block and identifying a difference (residual)between the predicted values and the current block. In this way, onlythe residual and parameters used to generate it need be added to thebitstream instead of including the entirety of the current block. Thistechnique may be referred to as inter prediction.

One of the parameters in inter prediction is a motion vector thatrepresents the spatial displacement of the previously coded blockrelative to the current block. The motion vector can be identified usinga method of motion estimation, such as a motion search. In motionsearch, a portion of a reference frame can be translated to a successionof locations to form a prediction block that can be subtracted from aportion of a current frame to form a series of residuals. The X and Ytranslations corresponding to the location having the smallest residualcan be selected as the motion vector. Bits representing the motionvector can be included in the encoded bitstream to permit a decoder toreproduce the prediction block and decode the portion of the encodedvideo bitstream associated with the motion vector.

For video compression schemes, the number of bits used to encode themotion vectors can be significant, especially for video streams encodedat lower data rates or higher compression ratios. To improve theencoding efficiency, a motion vector can be differentially encoded usinga reference motion vector, i.e., only the difference between the motionvector and the reference motion vector is encoded. In some instances,the reference motion vector can be selected from previously used motionvectors in the video stream, for example, the last non-zero motionvector from neighboring blocks. Selecting a previously used motionvector to encode a current motion vector can further reduce the numberof bits included in the encoded video bitstream and thereby reducetransmission and storage bandwidth requirements. Motion vectorreferencing modes allow a coding block to infer motion information frompreviously coded neighboring blocks.

In implementations of this disclosure, a reference motion vector can beselected from candidate motion vectors based on the distance between thereference block and the current block and the popularity of thereference motion vector. For example, the distance between the referenceblock and the current block can be based on the spacial displacementbetween the pixels in the previously coded block and the collocatedpixels in the current block, measured in the unit of pixels. Forexample, the popularity of the motion vector can be based on the amountof previously coded pixels that use the motion vector. The morepreviously coded pixels that use the motion vector, the higher theprobability of the motion vector. In one example, the popularity valueis the number of previously coded pixels that use the motion vector. Inanother example, the popularity value is a percentage of previouslycoded pixels within an area that use the motion vector.

Due to the proximity of the current block to the pixels in the referenceblock, it is likely in many cases that the current block has similarmotion characteristics to those pixels. Thus, a candidate motion vectorused in a reference block near the current block may closely resemblethe actual motion vector for the current block. Additionally, acandidate motion vector that is used by the most amount of pixels in areference block near the current block would further resemble the actualmotion vector for the current block. For this reason, the motion vectorof the candidate motion vectors with the highest popularity used in anearby reference block may be selected as the reference motion vectorfor the actual motion vector of the current block. Fewer bits can beused to code the actual motion vector by coding the small difference inmotion vectors, thus improving the overall coding efficiency. Other waysin which the selected motion vector may be used are discussedhereinafter.

The candidate motion vectors may be limited to spatial-temporalneighboring motion vectors. That is, the pool of candidate motionvectors may be selected from regions neighboring regions of the currentblock. In some video coding schemes, particularly those where videoframes are encoded out of order, it is desirable to include in the poolof candidate motion vectors motion information from video frames in thedistant past or future. Encoding video frames can out of order mayoccur, for example, in the coding of so-called “alternate referenceframes” that are not temporally neighboring to the frames codedimmediately before or after them. An alternate reference frame may be asynthesized frame that does not occur in the input video stream or is aduplicate frame to one in the input video stream that is used forprediction and is generally not displayed following decoding. Such aframe can resemble a video frame in the non-adjacent future. Anotherexample in which out of order encoding may occur is through the use of aso-called “golden reference frame,” which is a reconstructed video framethat may or may not be neighboring to a current video frame and isstored in memory for use as a reference frame until replaced, e.g., by anew golden reference frame.

Herein, alternate reference frames and golden reference frames (alsocalled alternate frames and golden frames), in addition to adjacentvideo frames are used to infer motion vectors for a block of a frame ofvideo data using pixels from the non-adjacent or adjacent video framesto predict reconstructed pixels spatially near the block to bepredicted. Other details are described herein after first describing anenvironment in which the disclosure may be implemented.

FIG. 1 is a schematic of a video encoding and decoding system 100 inwhich aspects of the disclosure can be implemented. An exemplarytransmitting station 102 can be, for example, a computer having aninternal configuration of hardware including a processor such as acentral processing unit (CPU) 104 and a memory 106. CPU 104 is acontroller for controlling the operations of transmitting station 102.CPU 104 can be connected to the memory 106 by, for example, a memorybus. Memory 106 can be read only memory (ROM), random access memory(RAM) or any other suitable memory device. Memory 106 can store data andprogram instructions that are used by CPU 104. Other suitableimplementations of transmitting station 102 are possible. For example,the processing of transmitting station 102 can be distributed amongmultiple devices.

A network 108 connects transmitting station 102 and a receiving station110 for encoding and decoding of the video stream. Specifically, thevideo stream can be encoded in transmitting station 102 and the encodedvideo stream can be decoded in receiving station 110. Network 108 canbe, for example, the Internet. Network 108 can also be a local areanetwork (LAN), wide area network (WAN), virtual private network (VPN), acellular telephone network or any other means of transferring the videostream from transmitting station 102 to, in this example, receivingstation 110.

Receiving station 110 can, in one example, be a computer having aninternal configuration of hardware including a processor such as a CPU112 and a memory 114. CPU 112 is a controller for controlling theoperations of receiving station 110. CPU 112 can be connected to memory114 by, for example, a memory bus. Memory 114 can be ROM, RAM or anyother suitable memory device. Memory 114 can store data and programinstructions that are used by CPU 112. Other suitable implementations ofreceiving station 110 are possible. For example, the processing ofreceiving station 110 can be distributed among multiple devices.

A display 116 configured to display a video stream can be connected toreceiving station 110. Display 116 can be implemented in various ways,including by a liquid crystal display (LCD), a cathode-ray tube (CRT),or a light emitting diode display (LED), such as an OLED display.Display 116 is coupled to CPU 112 and can be configured to display arendering 118 of the video stream decoded in receiving station 110.

Other implementations of the encoder and decoder system 100 are alsopossible. For example, one implementation can omit network 108 and/ordisplay 116. In another implementation, a video stream can be encodedand then stored for transmission at a later time by receiving station110 or any other device having memory. In one implementation, receivingstation 110 receives (e.g., via network 108, a computer bus, or somecommunication pathway) the encoded video stream and stores the videostream for later decoding. In another implementation, additionalcomponents can be added to the encoder and decoder system 100. Forexample, a display or a video camera can be attached to transmittingstation 102 to capture the video stream to be encoded.

FIG. 2 is a diagram of an example video stream 200 to be encoded anddecoded. Video stream 200 (also referred to herein as video data)includes a video sequence 204. At the next level, video sequence 204includes a number of adjacent frames 206. While three frames aredepicted in adjacent frames 206, video sequence 204 can include anynumber of adjacent frames. Adjacent frames 206 can then be furthersubdivided into individual frames, e.g., a single frame 208. Each frame208 can capture a scene with one or more objects, such as people,background elements, graphics, text, a blank wall, or any otherinformation.

At the next level, single frame 208 can be divided into a set of blocks210, which can contain data corresponding to, in some of the examplesdescribed below, a 8×8 pixel group in frame 208. Block 210 can also beof any other suitable size such as a block of 16×8 pixels, a block of8×8 pixels, a block of 16×16 pixels, a block of 4×4 pixels, or of anyother size. Unless otherwise noted, the term ‘block’ can include amacroblock, a subblock (i.e., a subdivision of a macroblock), a segment,a slice, a residual block or any other portion of a frame. A frame, ablock, a pixel, or a combination thereof can include displayinformation, such as luminance information, chrominance information, orany other information that can be used to store, modify, communicate, ordisplay the video stream or a portion thereof.

FIG. 3 is a block diagram of an encoder 300 in accordance withimplementations of this disclosure. Encoder 300 can be implemented, asdescribed above, in transmitting station 102 such as by providing acomputer software program stored in memory 106, for example. Thecomputer software program can include machine instructions that, whenexecuted by CPU 104, cause transmitting station 102 to encode video datain the manner described in FIG. 3. Encoder 300 can also be implementedas specialized hardware in, for example, transmitting station 102.Encoder 300 has the following stages to perform the various functions ina forward path (shown by the solid connection lines) to produce anencoded or a compressed bitstream 320 using input video stream 200: anintra/inter prediction stage 304, a transform stage 306, a quantizationstage 308, and an entropy encoding stage 310. Encoder 300 may include areconstruction path (shown by the dotted connection lines) toreconstruct a frame for encoding of future blocks. In FIG. 3, encoder300 has the following stages to perform the various functions in thereconstruction path: a dequantization stage 312, an inverse transformstage 314, a reconstruction stage 316, and a loop filtering stage 318.Other structural variations of encoder 300 can be used to encode videostream 200.

When video stream 200 is presented for encoding, each frame 208 withinvideo stream 200 can be processed in units of blocks. Referring to FIG.3, at intra/inter prediction stage 304, each block can be encoded usingeither intra prediction (i.e., within a single frame) or interprediction (i.e. from frame to frame). In either case, a predictionblock can be formed. The prediction block is then subtracted from theblock to produce a residual block (also referred to herein as residual).

Intra prediction (also referred to herein as intra-prediction orintra-frame prediction) and inter prediction (also referred to herein asinter-prediction or inter-frame prediction) are techniques used inmodern image/video compression schemes. In the case of intra-prediction,a prediction block can be formed from samples in the current frame thathave been previously encoded and reconstructed. In the case ofinter-prediction, a prediction block can be formed from samples in oneor more previously constructed reference frames, such as the last frame(i.e., the adjacent frame immediately before the current frame), thegolden frame or the constructed or alternate frame described above.

The prediction block is then subtracted from the current block. Thedifference, or residual, is then encoded and transmitted to decoders.Image or video codecs may support many different intra and interprediction modes; each block may use one of the prediction modes toobtain a prediction block that is most similar to the block to minimizethe information to be encoded in the residual so as to re-create theblock. The prediction mode for each block of transform coefficients canalso be encoded and transmitted so a decoder can use the same predictionmode(s) to form prediction blocks in the decoding and reconstructionprocess.

The prediction mode may be selected from one of multipleintra-prediction modes. Alternatively, the prediction mode may beselected from one of multiple inter-prediction modes using one or morereference frames including, for example, last frame, golden frame,alternative reference frame, or any other reference frame in an encodingscheme. The bitstream syntax supports three categories of interprediction modes. The inter prediction modes can include, for example, amode (sometimes called ZERO_MV mode) in which a block from the samelocation within a reference frame as the current block is used as theprediction block; a mode (sometimes called a NEW_MV mode) in which amotion vector is transmitted to indicate the location of a block withina reference frame to be used as the prediction block relative to thecurrent block; or a mode (sometimes called a REF_MV mode comprisingNEAR_MV or NEAREST_MV mode) in which no motion vector is transmitted andthe current block uses the last or second-to-last non-zero motion vectorused by neighboring, previously coded blocks to generate the predictionblock. Inter-prediction modes may be used with any of the availablereference frames.

The reference motion vector coding modes analyze the available blocksmotion activity to produce a ranked reference motion vector candidatelist, allowing the size of the reference motion vector modes to bedynamically extended, a generalization of the fixed NEAREST_MV mode andNEAR_MV mode, and improving the efficacy of entropy coding. REF_MV moderepresents reference motion vector modes, including NEAR_MV andNEAREST_MV modes, as well as extended modes when the candidate motionvector list shows sufficient variation. The entropy coding system startswith NEW_MV mode, selecting the probability model according to thecontexts of the number of reference motion vectors found and, if aneighboring block with matched reference frame is found, determining thelikelihood that the neighboring block is coded in NEW_MV mode. If thecoding mode is not NEW_MV mode, the codec continues to code if the modeis ZERO_MV mode. The codec selects the mode probability model accordingto the contexts when the collocated block in the previous frame is usinga zero motion vector, or close to zero motion vector (i.e., less than 1full pixel in both row and column components), or when the spatialneighboring blocks are using zero, or close to zero, motion vectors. Ifthe coding mode is not ZERO_MV mode, the codec decides between theREF_MV modes. The associated context is how many reference motionvectors checked into the candidate list and the weighing coefficients.The REF_MV mode syntax suggests that the effective motion vector comesfrom referring neighboring blocks coding information. Given thereference frame, a list of such possible reference motion vectorcandidates includes reference motion vector candidates that are rankedaccording to their relative distance to the current block and theoverlapped amount. The codec needs to decode which one in the list topick and use. The codec can select the candidate motion vector among thecandidate motion vectors in the list that would provide the bestprediction and send the index associated with that particular candidatemotion vector to the decoder.

Next, still referring to FIG. 3, transform stage 306 transforms theresidual into a block of transform coefficients in, for example, thefrequency domain. Examples of block-based transforms include theKarhunen-Loève Transform (KLT), the Discrete Cosine Transform (DCT),Walsh-Hadamard Transform (WHT), the Singular Value DecompositionTransform (SVD) and the Asymmetric Discrete Sine Transform (ADST). Inone example, the DCT transforms the block into the frequency domain. Inthe case of DCT, the transform coefficient values are based on spatialfrequency, with the lowest frequency (e.g., DC) coefficient at thetop-left of the matrix and the highest frequency coefficient at thebottom-right of the matrix.

Quantization stage 308 converts the block of transform coefficients intodiscrete quantum values, which are referred to as quantized transformcoefficients, using a quantizer value or quantization level. Thequantized transform coefficients are then entropy encoded by entropyencoding stage 310. The entropy-encoded coefficients, together withother information used to decode the block, which can include forexample the type of prediction used, motion vectors and quantizationvalue, are then output to compressed bitstream 320. Compressed bitstream320 can be formatted using various techniques, such as variable lengthencoding (VLC) and arithmetic coding. Compressed bitstream 320 can alsobe referred to as an encoded video stream and the terms will be usedinterchangeably herein.

The reconstruction path in FIG. 3 (shown by the dotted connection lines)can be used to provide both encoder 300 and a decoder 400 (describedbelow) with the same reference frames to decode compressed bitstream320. The reconstruction path performs functions that are similar tofunctions that take place during the decoding process that are discussedin more detail below, including dequantizing the quantized transformcoefficients at dequantization stage 312 to generate dequantizedtransform coefficients and inverse transforming the dequantizedtransform coefficients at inverse transform stage 314 to produce aderivative residual block (i.e., derivative residual). At reconstructionstage 316, the prediction block that was predicted at intra/interprediction stage 304 can be added to the derivative residual to create areconstructed block. In some implementations, loop filtering stage 318can be applied to the reconstructed block to reduce distortion such asblocking artifacts.

Other variations of encoder 300 can be used. For example, anon-transform based encoder 300 can quantize the residual block directlywithout transform stage 304. In another implementation, an encoder 300can have quantization stage 308 and dequantization stage 312 combinedinto a single stage.

FIG. 4 is a block diagram of a decoder 400 in accordance withimplementations of this disclosure. Decoder 400 can be implemented, forexample, in receiving station 110, such as by providing a computersoftware program stored in memory for example. The computer softwareprogram can include machine instructions that, when executed by CPU 112,cause receiving station 110 to decode video data in the manner describedin FIG. 4. Decoder 400 can also be implemented as specialized hardwareor firmware in, for example, transmitting station 102 or receivingstation 110. Decoder 400, similar to the reconstruction path of encoder300 discussed above, includes in one example the following stages toperform various functions to produce an output video stream 416 fromcompressed bitstream 320: an entropy decoding stage 402, adequantization stage 404, an inverse transform stage 406, an intra/interprediction stage 408, a reconstruction stage 410, a loop filtering stage412, and a deblocking filtering stage 414. Other structural variationsof decoder 400 can be used to decode compressed bitstream 320.

When compressed bitstream 320 is presented for decoding, the dataelements within compressed bitstream 320 can be decoded by the entropydecoding stage 402 (using, for example, arithmetic coding) to produce aset of quantized transform coefficients. Dequantization stage 404dequantizes the quantized transform coefficients and inverse transformstage 406 inverse transforms the dequantized transform coefficients toproduce a derivative residual that can be identical to that created byreconstruction stage 316 in encoder 300. Using header informationdecoded from compressed bitstream 320, decoder 400 can use intra/interprediction stage 408 to create the same prediction block as was createdin encoder 300, e.g., at intra/inter prediction stage 304. In the caseof inter prediction, the reference frame from which the prediction blockis generated may be transmitted in the bitstream or constructed by thedecoder using information contained within the bitstream.

At reconstruction stage 410, the prediction block can be added to thederivative residual to create a reconstructed block that can beidentical to the block created by reconstruction stage 316 in encoder300. In some implementations, loop filtering stage 412 can be applied tothe reconstructed block to reduce blocking artifacts. Deblockingfiltering stage 414 can be applied to the reconstructed block to reduceblocking distortion, and the result is output as output video stream416. Output video stream 416 can also be referred to as a decoded videostream and the terms will be used interchangeably herein.

Other variations of decoder 400 can be used to decode compressedbitstream 320. For example, decoder 400 can produce output video stream416 without deblocking filtering stage 414.

FIG. 5 is a flow diagram showing a process 500 for encoding a videostream using reference motion vectors in accordance with animplementation of this disclosure. Process 500 can be implemented in anencoder such as encoder 300 (shown in FIG. 3) and can be implemented,for example, as a software program that can be executed by computingdevices such as transmitting station 102 or receiving station 110 (shownin FIG. 1). For example, the software program can includemachine-readable instructions that can be stored in a memory such asmemory 106 or memory 114, and that can be executed by a processor, suchas CPU 104, to cause the computing device to perform process 500.

Process 500 can be implemented using specialized hardware or firmware.Some computing devices can have multiple memories, multiple processors,or both. The steps of process 500 can be distributed using differentprocessors, memories, or both. Use of the terms “processor” or “memory”in the singular encompasses computing devices that have one processor orone memory as well as devices that have multiple processors or multiplememories that can each be used in the performance of some or all of therecited steps. For simplicity of explanation, process 500 is depictedand described as a series of steps. However, steps in accordance withthis disclosure can occur in various orders and/or concurrently.Additionally, steps in accordance with this disclosure may occur withother steps not presented and described herein. Furthermore, not allillustrated steps may be required to implement a method in accordancewith the disclosed subject matter.

Process 500 assumes that a stream of video data having multiple frames,each having multiple blocks, is being encoded using a video encoder suchas video encoder 300 executing on a computing device such astransmitting station 102. The video data or stream can be received bythe computing device in any number of ways, such as by receiving thevideo data over a network, over a cable, or by reading the video datafrom a primary memory or other storage device, including a disk drive orremovable media such as a CompactFlash (CF) card, Secure Digital (SD)card, or any other device capable of communicating video data. In someimplementations, video data can be received from a video cameraconnected to the computing device operating the encoder. At least someof the blocks within frames are encoded using inter prediction asdescribed in more detail below.

At step 502, process 500 identifies candidate motion vectors frompreviously coded blocks in the video stream. Process 500 analyzes themotion activity of previously coded blocks to produce a list of rankedcandidate motion vectors, employing an efficient reference motion vectorranking system, and provides a dynamic motion referencing mode thatfully accounts for the available motion vector candidate list, whichallows the number of the reference motion vector modes to be dynamicallyextended or shortened and improves the efficacy of entropy coding. Thepool of the candidate motion vectors can be dynamically extended orshortened according to the neighboring reference block condition, i.e.,how the list of candidate motion vectors is created. The previouslycoded blocks in the video stream can include any block encoded usinginter-prediction before the current block, such as a block from apreviously coded frame or a block from the same frame as the currentblock that has been encoded before the current block. For example, insome encoding/decoding (codec) schemes such as ones that code in rasterscan order, the previously coded blocks can include a block above, tothe left, or to the above-left of the current block in the same frame.The previously coded blocks can also include, for example, a block fromthe immediately previous frame (i.e., last frame), a block from thegolden frame (described at intra/inter prediction stage 304), a blockfrom any other reference frame, or any combination thereof. Desirably,however, the candidate motion vectors are obtained from previously codedblocks that correspond in some way to the current block based on thetheory that such blocks, due to the proximity of their pixels to thecurrent block, are likely to have similar motion characteristics to thecurrent block.

Blocks may be predicted from a single reference frame or compound, suchas two, reference frames. For blocks with a single reference frame,process 500 identifies candidate motion vectors from the nearest spatialneighboring blocks that are associated with the reference frame. Thecandidate motion vectors may include the motion vector reference blockswithin a reference coverage area, such as the motion vector from a blockabove the current block, the motion vector from a block to the left ofthe current block, the motion vector from a block to the above-right ofthe current block, and the motion vector of a collocated block in theprevious frame.

Process 500 determines, for each candidate motion vector, a distancebetween the previously coded block associated with the candidate motionvector and the current block 504. The candidate motion vectors areranked based on the distance, from nearest to furthest, from thepreviously coded block and the overlapped length between the previouslycoded block and the current block. The nearest candidate motion vectorfrom the block above the current block, the nearest candidate motionvector from a block to the left of the current block, and the nearestcandidate motion vector from the above-right of the current block areconsidered to be category 1 candidate motion vectors. Due to theproximity of the previously coded block to the current block, it ispossible that the current block has similar motion characteristics asthe previously coded block. The remaining candidate motion vectors areconsidered to be category 2 candidate motion vectors. Category 1candidate motion vectors always assume a higher priority than category 2candidate motion vectors.

Process 500 determines, for each candidate motion vector within category1 and category 2, a popularity value based on a set of previously codedpixel values associated with the particular candidate motion vector 504.Within each category, the candidate motion vectors are ranked accordingto their popularity value, with category 1 candidate motion vectorsalways ranking higher than category 2 candidate motion vectors. Thelarger the number of previously coded pixel values associated with thecandidate motion vector, the higher the popularity value of thecandidate motion vector. At step 508, a reference motion vector can beselected from the candidate motion vectors identified in step 504, basedon the candidate motion vector nearest the current block and having thehighest popularity value. The motion vector of the current block can beencoded using the reference motion vector in step 510 before processingbegins again for the next block of the current frame. In addition, thecurrent block can be encoded according to the process described withrespect to FIG. 3.

Blocks can be predicted from two reference frames, where the predictionwill be the average of the two frames. In some cases, the average can bea weighted average. For blocks with compound reference frames, process500 finds the reference blocks that share the most reference frames withthe current block. Process 500 identifies and assigns a higher priorityto candidate motion vectors from neighboring reference blocks that sharethe same reference frame combination. When there are less than twoidentified candidate motion vectors, process 500 identifies additionalcombinations of candidate motion vectors from blocks with singlereference frames, as described above.

Process 500 determines, for each candidate motion vector, a distancebetween the previously coded block associated with the candidate motionvector and the current block 504. For example, the distance between thepreviously coded block associated with the candidate motion vector andthe current block can be based on the spacial displacement between thepixels in the previously coded block and the collocated pixels in thecurrent block, measured in the unit of pixels. The candidate motionvectors are ranked based on the distance, from nearest to furthest, fromthe previously coded block to the current block and the overlappedlength between the previously coded block and the current block. Thenearest candidate motion vector from the block above the current block,the nearest candidate motion vector from a block to the left of thecurrent block, and the nearest candidate motion vector from theabove-right of the current block are considered to be category 1candidate motion vectors. The remaining candidate motion vectors areconsidered to be category 2 candidate motion vectors.

Process 500 determines, for each candidate motion vector identified instep 502, a popularity value based on a set of previously coded pixelvalues associated with the candidate motion vector 506. A popularityvalue is determined for each compound candidate motion vector, eachcategory 1 candidate motion vector, and each category 2 candidate motionvector. The candidate motion vectors are then ranked according to apriority value based on their popularity value, with candidate motionvectors that share the same compound reference frames having a higherpriority than category 1 candidate motion vectors and category 1candidate motion vectors having a higher priority than category 2candidate motion vectors. The larger the number of previously codedpixel values associated with the candidate motion vector, the higher thepopularity value of the candidate motion vector. At step 508, areference motion vector can be selected from the candidate motionvectors identified in step 506, based on the candidate motion vectorhaving the highest priority. The motion vector of the current block canbe encoded using the reference motion vector in step 510 beforeprocessing begins again for the next block of the current frame. Inaddition, the current block can be encoded according to the processdescribed with respect to FIG. 3. Using the most popular candidatemotion vector, i.e., the candidate motion vector used by the most pixelsin the reference coverage area comprising neighboring blocks, reducesthe overall number of motion vectors that need to be encoded, improvingthe prediction accuracy of reference motion vectors and providing afurther level of optimization and bandwidth efficiency.

FIG. 6 is a diagram of a frame 600 including a current block 602 used toexplain the process of FIG. 5. Frame 600 includes blocks that have beenencoded before current block 602, such as the shaded blocks 604 to theleft of or above current block 602 in FIG. 6. In this example whereencoding is performed in raster scan order, the candidate motion vectorsmay include the motion vector from a block 604A above current block 602,the motion vector from a block 604B to the left of current block 602 andthe motion vector from a block 604C to the above-left of current block602. If any of blocks 604A, 604B or 604C were not intra predicted, theywould not have a motion vector to contribute to the candidate motionvectors. The candidate motion vectors can also include motion vectorsfrom other frames as illustrated by FIGS. 10 and 11.

FIG. 10 is a diagram of a series 1000 of frames F₁, F₂ . . . F_(k−1),F_(k) of a first video stream in accordance with an implementation ofthis disclosure. Frame F_(k) is the current frame to be encodedfollowing encoding and reconstructing frames F₁, F₂ . . . F_(k−1). FrameF_(k) includes the current block referred to in FIG. 5, for example.Frame F_(k−1) is temporally adjacent to frame F_(k), while frames F₁ andF₂ are temporally non-adjacent to frame F_(k). A frame (e.g., areference frame) is temporally non-adjacent to another frame when theframes are separated within a temporal sequence of the plurality offrames of the video stream by at least one frame. Among these videoframes, reconstructed frame F₂ may be stored as a golden reference frameas discussed above. Frame F_(k−1) is the reconstructed frame stored in a“last” reference frame buffer available for coding blocks of currentframe F_(k). When encoding frame F₂, frame F₁ was used as the “last”reference frame. A block that spatially corresponds to the current blockin last frame F_(k−1) may be used to obtain a motion vector for thecandidate motion vectors in step 502. That is, a motion vector used forthe prediction of the block in last frame F_(k−1) at the same pixellocation as the current block may be added to the candidate motionvectors. Motion vectors from other blocks in last frame F_(k−1), such asthose adjacent to the same pixel location as the current block, may alsobe used as candidate motion vectors in some cases. Pixel locations maybe designated by X- and Y-coordinates with the top-left pixel designatedas position (0,0) for example.

In the example of FIG. 10, frame F₂ is a golden frame available forinter prediction of blocks in current frame F_(k). Therefore, one ormore of the adjacent blocks to the current block in frame F_(k) mayrefer to frame F₂ such that its motion vector is included among thecandidate motion vectors. Further, one or more motion vectors used forthe prediction of the blocks in golden frame F₂ may also be added to thecandidate motion vectors. For example, a dominant motion vector of theframe could be selected. In some cases, motion vectors of interpredicted blocks in golden frame F₂ within a specified spatialneighborhood of, for example, the same pixel position as the currentblock may be used as candidate motion vectors. Flags may be associatedwith frame F_(k) (such as bits in its header) to indicate that a motionvector used in coding frame F₂ (e.g., against frame F₁) is available tosome blocks in frame F_(k) as a candidate motion vector.

FIG. 11 is a diagram of a series 1100 of frames F₁, A₁, F₂, . . . .F_(k−1), F_(k), . . . F_(k+m) of a second video stream in accordancewith an implementation of this disclosure. Series 1100 is similar toseries 1000 but includes an alternate reference frame A₁. Alternatereference frames may be purely constructed frames and, as such, may nothave the same dimensions as the remaining frames in series 1100. Forsimplicity in this explanation, it is assumed that frame A1 resembles afuture video frame F_(k+m). When encoding frame A₁, motion vectors maybe used against reference frame F₁, for example. When it is time toencode frame F_(k), a motion vector from encoded and reconstructed frameA₁ can now be selected and identified to be used as a candidate motionvectors in encoding one or more blocks in frame F_(k). As discussed inthe example given above with respect to golden frame F₂, a motion vectorto be included in the candidate motion vectors may be one associatedwith a spatially corresponding block of alternate reference frame A₁ orone associated with another nearby block. Further, any of the blocksadjacent to the current block in frame F_(k) may refer to frame A₁ suchthat the corresponding motion vector is included among the candidatemotion vectors.

FIG. 7 is a diagram of the current block 602 of FIG. 6 and a set 702 ofpreviously coded pixels. Set 702 can include, for example, two rows702A, 702B of pixels immediately above current block 602 and two columns702C, 702D of pixels to the immediate left of current block 602.However, other previously coded pixels (not shown in FIG. 7) can also beused. Rows 702A, 702B are associated with block 604A, while columns702C, 702D are associated with block 604B. Blocks, such as current block602 and previously coded blocks 604A, 604B, are shown in FIG. 7 to havea set of 8×8 pixels, which can be represented by an 8×8 matrix of pixelvalues. However, any other block size can be used. When a block isformed by a matrix of 16×16 pixels, for example, a 16×2 region from theblock above and a 2×16 region from the block to the left of the currentblock may be used. The number of pixels can be altered to include feweror more pixels.

A popularity value can be determined for each candidate motion vectorbased on how many previously coded pixels in the reference coverage areause that particular candidate motion vector. The more previously codedpixels that use a particular candidate motion vector, the higher thepopularity value will be for that particular candidate motion vector.Step 506 can be implemented, for example, at intra/inter predictionstage 306 of encoder 300 in FIG. 3, and one implementation is explainedusing FIG. 8.

FIG. 8 is a diagram of a set of predicted pixels for the set ofpreviously coded pixels 702 of FIG. 7. In FIG. 8, current block 602 ofcurrent frame 600 is being encoded. The set of predicted values isdetermined using a candidate motion vector (indicated generally by arrow802) identified at step 502. The set of previously coded pixels 702include, for example, two rows 702A, 702B and two columns 702C, 702Ddescribed above with reference to FIG. 7. To form the set of predictedvalues using candidate motion vector 802, rows 702A, 702B can bepredicted by rows 804A, 804B in a reference frame 800 and columns 702C,702D can be predicted by columns 804C, 804D in reference frame 800. Thatis, using the X- and Y-offsets provided by candidate motion vector 802,a set of predicted pixels represented by rows 804A, 804B and columns804C, 804D in reference frame 800 is identified. In FIG. 8, block 806 isshown in the same spatial position in reference frame 800 as currentblock 602 is in current frame 600 to illustrate the pixels of rows 804A,804B and columns 804C, 804D selected as the prediction pixels based onthe candidate motion vector.

The reference frame used for the identification of the prediction valuesin step 506 may be a temporally adjacent frame (such as last frameF_(k−1)) or a temporally non-adjacent frame (such as golden frame F₂ oralternate frame A₁). Desirably, each available reference frame is usedas part of a rate-distortion loop within an encoder that determines thebest coding mode for the current block by comparing the rate (e.g., thebit cost) of each coding mode with the resulting image distortion (e.g.,the change in image due to the coding) for each tested mode. However,since the candidate motion vectors may be generated using framesseparated by different temporal distances than the current frame and theparticular reference frame under consideration. Accordingly, step 506also includes scaling candidate motion vectors where needed, which isdescribed by reference again to FIGS. 10 and 11.

Scaling up or down a motion vector so that it may be applied as acandidate motion vector means adjusting its magnitude. The magnitude ofthe candidate can be scaled depending upon the results of comparing thetemporal distance and direction between the reference frame and theframe including the current block and the temporal distance anddirection used to form the candidate motion vector. The temporaldistance between frames can be determined by their respective positionsin the video stream. For example, when a candidate motion vector is amotion vector that was used to encode a block of frame F₂ against frameF₁, the magnitude of the motion vector can be used directly for encodingframe F_(k) against reference frame F_(k−1) since frames F_(k) andF_(k−1) are, like frames F₁ and F₂, a frame apart temporally (that is,they are adjacent frames in the frame sequence). In contrast, acandidate motion vector from F_(k−1) and previous coded blocks of F_(k)when a block of F_(k) was encoded using the golden frame, i.e., F₂ inthis FIGS. 10 and 11, needs to be scaled to take into account thedifference of temporal distance before being used to generate theprediction pixels in step 506. For example, a motion vector used inpreviously coding a block of current frame F_(k) against F_(k−1) will bescaled up using a factor proportional to k−2 to become a candidatemotion vector for generation of the prediction pixels when the currentblock, such as block 602 of FIG. 8, is in evaluation to be coded againstreference frame F₂. Scaling up or down a motion vector so that it may beapplied as a candidate motion vector means adjusting its magnitude. Themagnitude of the candidate can be scaled depending upon the results ofcomparing the temporal distance and direction between the referenceframe and the frame including the current block and the temporaldistance and direction used to form the candidate motion vector.

An alternate reference frame such as frame A₁ may be treated similarlyto other references frames, such as the last or golden reference frame.However, since an alternate reference frame may be constructed usingportions of multiple frames from multiple temporal positions in thevideo stream, techniques may be used to determine a temporal position inthe video stream that most closely matches the image data included inthe alternate frame.

This can be explained further using an example from FIG. 11 assuming theuse of a candidate motion vector used to predict a block in F_(k) fromframe A₁ to generate prediction pixels for analysis of the current blockusing reference frame F_(k−1). Because the distance in frames between F₁and A₁, where A₁ actually resembles frame F_(k+m), is k+m−1, the motionvector magnitude may be divided by k+m−1 before using the candidatemotion vector to generate the prediction pixels. This is because thedistance between frames F_(k) and F_(k−1) is less than the distancek+m−1, therefore the motion vector has to scaled down to approximate themotion between frames F_(k) and F_(k−1). In cases where the motionvectors are taken from frames temporally close together and used to forminter prediction pixels from frames with greater temporal separation,the motion vectors can be multiplied by a scaling factor proportional tothe ratio of the two temporal distances.

At step 508, a reference motion vector can be selected from thecandidate motion vectors identified at step 506. For blocks using asingle reference frame, the selection can be based on, for example,selecting the motion vector from the candidate motion vectors associatedwith the highest popularity value and associated with the referenceblock closest to the current block. For blocks using compound referenceframes, the selection can be based on, for example, the motion vectorfrom reference blocks that share the same reference frame combinationwith higher priority. When less than two candidate motion vectors areidentified, the candidate motion vectors in compound reference framescan be appended with the combinations of reference motion vectors ofsingle reference frames. Other selection criteria can also be used. Thereference motion vector can be used for further processing. Namely, themotion vector of the current block can be encoded using the referencemotion vector in step 510 before processing begins again for the nextblock of the current frame. In addition, the current block can beencoded according to the process described with respect to FIG. 3.

There are several ways to use the reference motion vector in encodingthe motion vector of the current block. For example, and as describedabove, process 500 may be part of a rate-distortion loop used to selectthe inter prediction mode for the current block to be encoded. As partof the rate-distortion loop, the actual motion vector for interprediction of the current block may be determined through a motionsearch according to any number of techniques. One use of the referencemotion vector may include using the reference motion vector as astarting parameter for the motion search algorithm based on thereasoning that the actual motion vector is likely to be close to thoseused in selecting the reference motion vector. A motion search mayalternatively be performed before or in parallel with process 500.

Whether or not the reference motion vector is incorporated into themotion search, step 510 may include using the reference motion vector todifferentially encode the actual motion vector. For example, adifference value can be calculated by subtracting the reference motionvector from the motion vector used to encode the current block. Thedifference value can be encoded and included in the video stream. Sincethe reference motion vector was formed using previously encoded anddecoded data, the same data can be available at a decoder to identifythe same reference motion vector as was used in forming the motionvector at the encoder, thus no motion vector is required to be encodedand transmitted for the current block. The decoded difference value canbe added to the reference motion vector identified by the decoder asdescribed below to form a motion vector to decode the current block.Note that the reference motion vector is associated with one of theavailable reference frames. Therefore, in the event the reference motionvector is associated with a reference frame that is different from thereference frame associated with the actual motion vector, the referencemotion vector may be scaled as described previously so as to generatethe difference between the reference motion vector and the actual motionvector. In some implementations, a separate indication of the referenceframe used would also be encoded into the bitstream.

In another implementation, the reference motion vector may be used tochoose a probability distribution to encode the magnitude of the motionvector used to encode the current block. In this implementation, bitscan be included in the video stream to identify the encoded magnitude ofthe motion vector and which predetermined probability distribution touse to form the motion vector based on the encoded magnitude. One ormore bits indicating which reference frame to use in decoding thecurrent block may also be included in the bitstream in some variations.Like its use in differential encoding, the reference motion vector mayalso be scaled to the extent it is desirable.

In an implementation, the reference motion vector may also be useddirectly in the encoding of the current block. This can occur, forexample, when a comparison of the rate-distortion value involved incoding the current block using the motion vector determined by themotion search is higher than that involved in coding the current blockusing the reference motion vector. In this comparison, the referenceframe used would desirably be the one used in selecting the referencemotion vector so no scaling is needed. In some cases, the decision as towhether or not to use the reference motion vector may be tied to thedifference between the reference motion vector and the motion vectorresulting from the search. When the difference is small (or zero), thedifference in prediction results for the reference frame resulting fromthe search using the reference motion vector versus the actual motionvector is also small (or zero). When the reference motion vector is useddirectly to encode the current block, no motion vector would need to beseparately encoded at step 510. Instead, one or more bits would beinserted into the bitstream in association with the current block toindicate use of the reference motion vector for encoding.

In each of the above ways, the use of a reference motion vector mayreduce the number of bits needed to represent the motion vector neededto decode an inter coded block. In some cases, the motion vector usedfor encoding the current frame would not be done separately. Using themost popular motion vector reduces the overall number of motion vectors,which in turn reduces the number of motion vectors that need to beencoded and placed in the bitstream. Bits may be inserted into frame,slice and/or block headers indicating whether reference motion vectorsare used and how they are used for encoding the current block. Whenapplicable, the motion vector found by the motion search or the motionvector differential and/or the reference frame used in encoding thecurrent block are also transmitted.

Regardless of the motion vector used for encoding, a prediction blockcan be determined based on a reference frame by applying a candidatemotion vector to the previously coded pixel values of the referenceframe. The prediction block can be subtracted from the current block toform a residual that can be further encoded according to the processingdescribed with respect to FIG. 3 and included in an encoded videobitstream.

FIG. 9 is a flow diagram of a process 900 for decoding an encoded videostream using reference motion vectors in accordance with implementationsof this disclosure. The decoder can reproduce the same candidate motionvectors that were constructed in process 500, shown in FIG. 5. Thedecoder can read the index of the bitstream to determine which referencemotion vector to use from the candidate motion vectors. Process 900 canbe implemented, for example, as a software program that may be executedby computing devices such as transmitting station 102 or receivingstation 110. For example, the software program can includemachine-readable instructions that may be stored in a memory such asmemory 106 or 114, and that, when executed by a processor, such as CPU104 or 112, may cause the computing device to perform process 900.Process 900 can be implemented using specialized hardware or firmware.As explained above, some computing devices may have multiple memories orprocessors, and the steps of process 900 can be distributed usingmultiple processors, memories, or both.

For simplicity of explanation, process 900 is depicted and described asa series of steps. However, steps in accordance with this disclosure canoccur in various orders and/or concurrently. Additionally, steps inaccordance with this disclosure may occur with other steps not presentedand described herein. Furthermore, not all illustrated steps may berequired to implement a method in accordance with the disclosed subjectmatter.

Desirably, process 900 substantially conforms to process 500. There aresome differences, however, that are pointed out in the followingdescription of process 900. Where steps are substantially similar tothose in process 500, reference will be made to the description above.

At step 902, the decoder determines whether the motion vector for thecurrent block was encoded using a reference motion vector, the referenceframe, and whether single or compound prediction was used. Thisinformation can be communicated by reading and decoding bits from anencoded video bitstream that indicate the use of a reference motionvector according to one of the techniques disclosed above. The encodedbitstream (or encoded video data) may have been received by decoder of acomputing device in any number of ways, such as by receiving the videodata over a network, over a cable, or by reading the video data from aprimary memory or other storage device, including a disk drive orremovable media such as a DVD, CompactFlash (CF) card, Secure Digital(SD) card, or any other device capable of communicating a video stream.Step 902 involves decoding at least a portion of the encoded videobitstream to extract the information regarding the motion vector for thecurrent block. This information can be included in a header associatedwith a current block or a frame header, for example. The information inthe one or more headers indicate to the decoder that the current blockis to be decoded using inter prediction and that the motion vector usedfor that inter prediction relies on the reference motion vector asdescribed previously. For example, information in the bitstream couldindicate that the actual motion vector used in encoding the currentblock was differentially encoded using the reference motion vector.Alternatively, information could indicate that the reference motionvector was used directly for encoding the current block.

When a reference motion vector was used in the encoder to encode themotion vector for the current block, process 900 advances to step 904 toidentify reference candidate motion vectors from the above and leftneighbor blocks of the same reference frame. The identified candidatereference motion vectors should be the same as those identified by theencoder in step 502, which may be accomplished by flags as describedpreviously and/or by a priori rules regarding the selection of candidatemotion vectors that are available to both the encoder and decoder basedon the position of the current block.

At step 906, the candidate reference motion vectors are ranked based onthe distance between the reference block and the current block and thepopularity of each candidate reference motion vector. The distancebetween the reference block and the current block is determined for eachcandidate reference motion vector. The popularity for each candidatereference motion vector is determined by a popularity value based on aset of previously coded pixel values associated with the particularcandidate reference motion vector. The most probable candidate referencemotion vector is assigned as the predicted motion vector for NEW_MVmode.

At step 908, two effective motion vectors are assigned to the REF_MVmode, comprising NEAREST_MV mode and NEAR_MV mode, based on theirrelative distance to the current block and the overlapped length betweenthe reference block and the current block and their popularity value. Ifthe list of candidate reference motion vectors includes more than twocandidate reference motion vectors, the number of reference motionvector modes is dynamically extended. Step 910 process 900 determineswhether to use NEW_MV mode or REF_MV mode. When REF_MV mode is selected,process 900 further determines whether to use NEAREST_MV mode or NEAR_MVmode.

Once the reference motion vector is selected, the motion vector used toencode the current block can be decoded using the selected referencemotion vector at step 912. The decoded motion vector may then be used todecode the current block according to the process of FIG. 4.

In one example of the implementation of step 912, if the referencemotion vector is used to differentially encode the actual motion vectorfor the current block, the decoder can decode the motion vector by, forexample, decoding an encoded difference value that can then be added tothe reference motion vector selected at step 910 to generate the actualmotion vector. Then, the actual motion vector may be used to decode thecurrent block using inter prediction. In other implementations, thereference motion vector can be used to identify a predeterminedprobability distribution, which can be used to decode a magnitude valueof the motion vector used to encode the current block before decodingthe current block using the motion vector. Similar to the discussion instep 510 of FIG. 5, this may involve scaling the reference motionvector. In other implementations, the reference motion vector may beused directly as the motion vector to decode the current block afterdecoding one or more bits indicating that the reference motion vectorshould be so used.

Once the motion vector and current block are decoded at step 912, thenext block may be processed. In the next block is inter coded, process900 may be repeated. A frame can be reconstructed from the blocksderived from reconstructed values by intra or inter prediction, or both.The output can be an output video stream, such as the output videostream 416 shown in FIG. 4.

According to the teachings herein, a reference motion vector may beselected so as to reduce the number of bits required to encode a motionvector determined by, for example, motion search techniques. Theteachings herein take advantage of temporal motion continuity to reducethe number of bits required to transmit motion vector information byreferring to motion vectors from adjacent and non-adjacent video frames.The decoder has all the information the encoder has to select thereference motion vector, allowing the selection of the reference motionvector without explicit transfer of further information.

The aspects of encoding and decoding described above illustrate someexemplary encoding and decoding techniques. However, it is to beunderstood that encoding and decoding, as those terms are used in theclaims, could mean compression, decompression, transformation, or anyother processing or change of data.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example” or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an implementation” or “oneimplementation” throughout is not intended to mean the same embodimentor implementation unless described as such.

Implementations of transmitting station 102 and/or receiving station 110(and the algorithms, methods, instructions, etc., stored thereon and/orexecuted thereby, including by encoder 300 and decoder 400) can berealized in hardware, software, or any combination thereof. The hardwarecan include, for example, computers, intellectual property (IP) cores,application-specific integrated circuits (ASICs), programmable logicarrays, optical processors, programmable logic controllers, microcode,microcontrollers, servers, microprocessors, digital signal processors orany other suitable circuit. In the claims, the term “processor” shouldbe understood as encompassing any of the foregoing hardware, eithersingly or in combination. The terms “signal” and “data” are usedinterchangeably. Further, portions of transmitting station 102 andreceiving station 110 do not necessarily have to be implemented in thesame manner.

Further, in one aspect, for example, transmitting station 102 orreceiving station 110 can be implemented using a general purposecomputer or general purpose processor with a computer program that, whenexecuted, carries out any of the respective methods, algorithms and/orinstructions described herein. In addition or alternatively, forexample, a special purpose computer/processor can be utilized which cancontain other hardware for carrying out any of the methods, algorithms,or instructions described herein.

Transmitting station 102 and receiving station 110 can, for example, beimplemented on computers in a video conferencing system. Alternatively,transmitting station 102 can be implemented on a server and receivingstation 110 can be implemented on a device separate from the server,such as a hand-held communications device. In this instance,transmitting station 102 can encode content using an encoder 300 into anencoded video signal and transmit the encoded video signal to thecommunications device. In turn, the communications device can thendecode the encoded video signal using a decoder 400. Alternatively, thecommunications device can decode content stored locally on thecommunications device, for example, content that was not transmitted bytransmitting station 102. Other suitable transmitting station 102 andreceiving station 110 implementation schemes are available. For example,receiving station 110 can be a generally stationary personal computerrather than a portable communications device and/or a device includingan encoder 300 may also include a decoder 400.

Further, all or a portion of implementations of the present disclosurecan take the form of a computer program product accessible from, forexample, a tangible computer-usable or computer-readable medium. Acomputer-usable or computer-readable medium can be any device that can,for example, tangibly contain, store, communicate, or transport theprogram for use by or in connection with any processor. The medium canbe, for example, an electronic, magnetic, optical, electromagnetic, or asemiconductor device. Other suitable mediums are also available.

The above-described embodiments, implementations and aspects have beendescribed in order to allow easy understanding of the present disclosureand do not limit the present disclosure. On the contrary, the disclosureis intended to cover various modifications and equivalent arrangementsincluded within the scope of the appended claims, which scope is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structure as is permitted under the law.

1. A method for encoding a video stream, comprising: identifying, forrespective previously coded blocks of a plurality of previously codedblocks, a candidate motion vector used to inter predict the previouslycoded block to define a plurality of candidate motion vectors;determining, using a processor, for each candidate motion vector of theplurality of candidate motion vectors, a distance between the previouslycoded block associated with the candidate motion vector and a currentblock of a current frame; determining, using the processor, for eachcandidate motion vector of the plurality of candidate motion vectors, apopularity value based on a set of previously coded pixels associatedwith the candidate motion vector; for encoding the current block usingcompound prediction, determining a first reference frame and a secondreference frame; ranking each candidate motion vector of the pluralityof candidate motion vectors within a list of candidate reference motionvectors according to a priority value for each candidate motion vector,the priority value based on the distance and the popularity valueassociated with the candidate motion vector, wherein for encoding thecurrent block using compound prediction, ranking each candidate motionvector comprises: determining reference blocks of the plurality ofpreviously coded blocks that share the first reference frame and thesecond reference frame: and assigning a higher priority to candidatemotion vectors from the reference blocks of the plurality of previouslycoded blocks that share a same combination of the first reference frameand the second reference frame within the list of candidate referencemotion vectors than combinations of candidate motion vectors from othersof the plurality of previously coded blocks encoded using respectivesingle reference frames of the first reference frame and the secondreference frame; selecting, based on the priority values, a referencemotion vector from the plurality of candidate motion vectors; andencoding a motion vector for the current block using the referencemotion vector.
 2. The method of claim 1, further comprising: assigning,as a predicted motion vector, a candidate motion vector within the listof candidate reference motion vectors having a highest priority value toa new motion vector coding mode that transmits a motion vector for interprediction of the current block; assigning, as a first reference motionvector, a candidate motion vector based on its position within the listof candidate reference motion vectors to a first reference motion vectorcoding mode that uses the first reference motion vector for interprediction of the current block; and assigning, as a second referencemotion vector, a candidate motion vector based on its position withinthe list of candidate reference motion vectors to a second referencemotion vector coding mode that uses the second reference motion vectorfor inter prediction of the current block, wherein selecting thereference motion vector comprises comparing a rate and image distortionof inter prediction of the current block using each of the new motionvector coding mode, the first reference motion vector coding mode, andthe second motion vector coding mode, and selecting an inter-predictionmode for encoding the current block based on the comparing.
 3. Themethod of claim 2, further comprising: transmitting an index associatedwith the reference motion vector to a decoder.
 4. The method of claim 1,further comprising: dynamically extending or dynamically shortening thelist of candidate reference motion vectors based on the plurality ofcandidate motion vectors.
 5. The method of claim 1, wherein at least oneof the plurality of previously coded blocks is in a reference frame thatis temporally separated from the current frame within a sequence of thevideo and the plurality of previously coded blocks includes at least oneof one or more blocks above the current block, one or more blocks to theleft of the current block, one or more blocks to the above-right of thecurrent block, or one or more blocks collocated with the current block.6. The method of claim 5, wherein the plurality of candidate motionvectors comprises a first set of candidate motion vectors and a secondset of candidate motion vectors, wherein each candidate motion vector ofthe first set of candidate motion vectors is associated with at leastone of the one or more blocks above the current block, the one or moreblocks to the left of the current block, or the one or more block to theabove-right of the current block, and wherein each candidate motionvector of the first set of candidate motion vectors is associated with ahigher priority value than each candidate motion vector of the secondset of candidate motion vectors.
 7. (canceled)
 8. (canceled)
 9. Themethod of claim 5, wherein the plurality of candidate motion vectorscomprises a first set of candidate motion vectors associated with a setof previously coded blocks of the plurality of previously coded blocks,wherein each previously coded block of the set of previously codedblocks associated with a plurality of reference frames, and wherein eachcandidate motion vector of the first set of candidate motion vectors isassociated with a higher priority value than each of the other candidatemotion vectors of the plurality of candidate motion vectors.
 10. Themethod of claim 9, wherein the plurality of candidate motion vectorsfurther comprises a second set of candidate motion vectors and a thirdset of candidate motion vectors on a condition that the first set ofcandidate motion vectors comprises less than two candidate motionvectors, wherein the second set of candidate motion vectors isassociated with at least one of the one or more blocks above the currentblock, the one or more blocks to the left of the current block, or theone or more blocks to the above-right of the current block, wherein eachcandidate motion vector of the first set of candidate motion vectors isassociated with a higher priority value than each candidate motionvector of the second set of candidate motion vectors and each candidatemotion vector of the third set of candidate motion vectors, and whereineach candidate motion vector of the second set of candidate motionvectors is associated with a higher priority value than each candidatemotion vector of the third set of candidate motion vectors.
 11. Themethod of claim 1, further comprising: encoding the current block into abitstream with at least one bit indicating an index associated with atleast one reference frame of the motion vector for the current block.12. The method of claim 1, wherein encoding the current block comprisestransmitting at least one bit in a bitstream indicating which of thefirst reference motion vector or the second reference motion vector isused as the motion vector to encode the current block.
 13. A method fordecoding an encoded video stream, comprising: determining, from bitsincluded in the encoded video stream, a reference frame on a conditionthat a current block to be decoded was encoded using one of a pluralityof inter-prediction modes, the current block comprising one of aplurality of blocks of a current frame of the encoded video stream;identifying, for respective previously decoded blocks of a plurality ofpreviously decoded blocks, a candidate motion vector used to interpredict the previously decoded block to define a plurality of candidatemotion vectors; determining, using a processor, for each candidatemotion vector of the plurality of candidate motion vectors, a distancebetween the previously decoded block associated with the candidatemotion vector and the current block; determining, using the processor,for each candidate motion vector of the plurality of candidate motionvectors, a popularity value based on a set of previously decoded pixelsassociated with the candidate motion vector; ranking each candidatemotion vector of the plurality of candidate motion vectors within a listof candidate reference motion vectors according to a priority value foreach candidate motion vector, the priority value based on the distanceand the popularity value associated with the candidate motion vector;determining a predicted motion vector from the plurality of candidatemotion vectors, the predicted motion vector associated with thecandidate motion vector having the highest popularity value; determininga first effective motion vector and a second effective motion vectorfrom the plurality of candidate motion vectors, the first effectivemotion vector and the second effective motion vector associated with thecandidate motion vectors having the shortest distance; selecting areference motion vector from the predicted motion vector, the firsteffective motion vector, and the second effective motion vector based onthe one of a plurality of inter-prediction modes used to encode thecurrent block; and decoding the current block using the reference motionvector, wherein the current block was encoded using compound prediction,and the reference frame is a first reference frame of at least tworeference frames including the first reference frame and a secondreference frame, the method further comprising: determining referenceblocks of the plurality of previously decoded blocks that share the atleast two reference frames, wherein ranking each candidate motion vectorcomprises: assigning a higher priority to candidate motion vectors fromthe reference blocks of the plurality of previously decoded blocks thatshare a same combination of the at least two reference frames within thelist of candidate reference motion vectors than combinations ofcandidate motion vectors from others of the plurality of previouslydecoded blocks encoded using respective single reference frames of theat least two reference frames.
 14. The method of claim 13, whereindecoding the current block comprises decoding bits within the encodedvideo stream indicating how the reference motion vector was used. 15.The method of claim 13, further comprising: assigning the firsteffective motion vector to a first reference motion vector coding modeof the plurality of inter-prediction modes that uses the first effectivemotion vector for inter prediction; and assigning the second effectivemotion vector to a second reference motion vector coding mode of theplurality of inter-prediction modes that uses the second effectivemotion vector for inter prediction, wherein the one of the plurality ofinter-prediction modes comprises the first reference motion vectorcoding mode or the second reference motion vector coding mode, anddecoding the current block comprises: decoding the current block usingthe reference motion vector as the motion vector by determining aprediction block within the reference frame using the motion vector andcombining a residual from the encoded video stream for the current blockwith the prediction block.
 16. (canceled)
 17. (canceled)
 18. The methodof claim 13, wherein the plurality of previously decoded blocks includesone or more blocks above the current block, one or more blocks to theleft of the current block, one or more blocks to the above-right of thecurrent block, or one or more blocks collocated with the current blockin a frame separated from the current frame within a temporal sequenceof the video stream by at least one frame.
 19. The method of claim 18,wherein the plurality of candidate motion vectors comprises a first setof candidate motion vectors and a second set of candidate motionvectors, wherein each candidate motion vector of the first set ofcandidate motion vectors is associated with the one or more blocks abovethe current block, the one or more blocks to the left of the currentblock, or the one or more block to the above-right of the current block,and wherein each candidate motion vector of the first set of candidatemotion vectors is associated with a higher priority value than eachcandidate motion vector of the second set of candidate motion vectors.20. The method of claim 13, wherein the plurality of previously decodedblocks comprises one or more blocks from a previously decoded frame orone or more blocks from the current frame.
 21. The method of claim 18,wherein the plurality of candidate motion vectors comprises a first setof candidate motion vectors associated with a set of previously codedblocks of the plurality of previously decoded blocks, the set ofpreviously coded blocks associated with a plurality of reference frames,wherein each previously coded block of the set of previously codedblocks comprises the same plurality of reference frames, and whereineach candidate motion vector of the first set of candidate motionvectors is associated with a higher priority value than each of theother candidate motion vectors of the plurality of candidate motionvectors.
 22. The method of claim 21, wherein the plurality of candidatemotion vectors further comprises a second set of candidate motionvectors and a third set of candidate motion vectors on a condition thatthe first set of candidate motion vectors comprises less than twocandidate motion vectors, wherein the second set of candidate motionvectors is associated with the one or more blocks above the currentblock, the one or more blocks to the left of the current block, or theone or more blocks to the above-right of the current block, wherein eachcandidate motion vector of the first set of candidate motion vectors isassociated with a higher priority value than each candidate motionvector of the second set of candidate motion vectors and each candidatemotion vector of the third set of candidate motion vectors, and whereineach candidate motion vector of the second set of candidate motionvectors is associated with a higher priority value than each candidatemotion vector of the third set of candidate motion vectors.
 23. Anapparatus for encoding a video stream, comprising: a processorconfigured to execute instructions to: identify, for respectivepreviously coded blocks of a plurality of previously coded blocks, acandidate motion vector used to inter predict the previously coded blockto define a plurality of candidate motion vectors; determine, for eachcandidate motion vector of the plurality of candidate motion vectors, adistance between the previously coded block associated with thecandidate motion vector and a current block of a current frame;determine, for each candidate motion vector of the plurality ofcandidate motion vectors, a popularity value based on a set ofpreviously coded pixels associated with the candidate motion vector; forencoding the current block using compound prediction, determine a firstreference frame and a second reference frame; rank each candidate motionvector of the plurality of candidate motion vectors within a list ofcandidate reference motion vectors according to a priority value foreach candidate motion vector, the priority value based on the distanceand the popularity value associated with the candidate motion vector,wherein for encoding the current block using compound prediction, theprocessor is configured to rank each candidate motion vector by:determining reference blocks of the plurality of previously coded blocksthat share the first reference frame and the second reference frame: andassigning a higher priority to candidate motion vectors from thereference blocks of the plurality of previously coded blocks that sharea same combination of the first reference frame and the second referenceframe within the list of candidate reference motion vectors thancombinations of candidate motion vectors from others of the plurality ofpreviously coded blocks encoded using respective single reference framesof the first reference frame and the second reference frame; select,based on the priority values, a reference motion vector from theplurality of candidate motion vectors; and encode a motion vector forthe current block using the reference motion vector.
 24. The apparatusof claim 23, wherein the processor is configured to encode the motionvector for the current block by at least one of: differentially encodingthe motion vector using the reference motion vector; or including atleast one bit within a bitstream in association with the current blockindicating that the current block was encoded using a reference motionvector.
 25. The method of claim 1, wherein ranking each candidate motionvector of the plurality of candidate motion vectors within a list ofcandidate reference motion vectors comprises: assigning each candidatemotion vector of the plurality of candidate motion vectors to one or afirst category or a second category of the list of candidate referencemotion vectors based on the distance; ranking a respective candidatemotion vector within the first category based on the popularity valueassociated with the respective candidate motion vector; ranking arespective candidate motion vector within the second category based onthe popularity value associated with the respective candidate motionvector; and arranging the list of candidate reference motion vectorssuch that the ranked candidate motion vectors of the first category havea higher priority than the ranked candidate motion vectors of the secondcategory.
 26. The method of claim 25, wherein assigning each candidatemotion vector of the plurality of candidate motion vectors to one or thefirst category or the second category comprises: assigning a candidatemotion vector of one or more blocks above the current block to the firstcategory; assigning a candidate motion vector of one or more block tothe left of the current block to the first category; assigning acandidate motion vector of a block to the above-right of the currentblock to the first category; and assigning a candidate motion vector ofone or more blocks collocated with the current block in a frameseparated from the current frame within a temporal sequence of the videostream to the second category.
 27. (canceled)
 28. (canceled)
 29. Themethod according to claim 1, further comprising: assigning, as apredicted motion vector, a candidate motion vector within the list ofcandidate reference motion vectors having a highest priority value to anew motion vector coding mode that transmits a motion vector for interprediction of the current block; assigning, as a first reference motionvector, a candidate motion vector based on its position within the listof candidate reference motion vectors to a first reference motion vectorcoding mode that uses the first reference motion vector for interprediction of the current block; and assigning, as a second referencemotion vector, a candidate motion vector based on its position withinthe list of candidate reference motion vectors to a second referencemotion vector coding mode that uses the second reference motion vectorfor inter prediction of the current block, wherein selecting thereference motion vector comprises selecting an inter-prediction mode forencoding the current block from the new motion vector coding mode, thefirst reference motion vector coding mode, the second motion vectorcoding mode, and a zero motion vector coding mode that uses a block froma same location within a reference frame as the current block as aprediction block.
 30. The apparatus of claim 23, wherein the processoris configured to: assign, as a predicted motion vector, a candidatemotion vector within the list of candidate reference motion vectorshaving a highest priority value to a new motion vector coding mode thattransmits a motion vector for inter prediction of the current block;assign, as a first reference motion vector, a candidate motion vectorbased on its position within the list of candidate reference motionvectors to a first reference motion vector coding mode that uses thefirst reference motion vector for inter prediction of the current block;and assign, as a second reference motion vector, a candidate motionvector based on its position within the list of candidate referencemotion vectors to a second reference motion vector coding mode that usesthe second reference motion vector for inter prediction of the currentblock, wherein: the processor is configured to select the referencemotion vector by selecting an inter-prediction mode for encoding thecurrent block from the new motion vector coding mode, the firstreference motion vector coding mode, the second reference motion vectorcoding mode, and a zero motion vector coding mode that uses a block froma same location within a reference frame as the current block as aprediction block, and the processor is configured to encode the motionvector for the current block by encoding the current block using theselected inter-prediction mode.